Opus: a Systematic Search Algorithm and Its Application to Categorical Attribute-value Data- Driven Machine Learning

OPUS is a branch and bound search algorithm that enables efficient systematic search through spaces in which the order in which the search operators are applied is not significant. OPUS achieves this by maximising the effect of each pruning action. While it is not possible to guarantee in the general case that any pruning shall occur, when pruning is possible, its effect is maximised. Experimental application of OPUS in data-driven machine learning demonstrates that NP hard search problems in which it is not possible to guarantee a solution in reasonable time can be solved for real world data within acceptable time frames. Indeed, OPUS is demonstrated to enable systematic search of extremely large search spaces in less time than is taken by common heuristic machine learning search algorithms. The use of systematic search in concept learning enables better experimental comparison of alternative inductive biases than was previously possible as the precise inductive bias can be described and manipulated. Such comparison reveals that the explicit biases used in some previous concept learning research have interacted with the search heuristics employed to produce implicit biases that frequently out-perform the explicit biases. Explicit formulation of these implicit biases would enable their systematic study and refinement with the potential of improving the predictive accuracy of the hypotheses inferred through machine learning.